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PATENT APPLICATION 



AUDIENCE ATTENTION AND RESPONSE EVALUATION 



CROSS-REFERENCE TO RELATED APPLICATIONS 



This application claims the benefit of the filing of U.S. Provisional Patent Application Serial No. 
60/208,001 , entitled "System for Tracking Audience Response to Advertising", filed on May 26, 2000, 
U.S. Provisional Patent Application Serial No. 60/208,003, entitled "Method for Testing Audience 
Attention and Response to Advertising in the Home or Place of Business from a Distance", filed on 
May 26, 2000, and U.S. Provisional Patent Application Serial No. 60/208,004, entitled "Method for 
Automatically Recording and Displaying Audience Attention to Video", filed on May 26, 2000, and the 
specifications thereof are incorporated herein by reference. 



A portion of the disclosure of this patent document and of the related applications listed above 
contains material that is subject to copyright protection. The owner has no objection to the facsimile 
reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and 
Trademark Office patent file or records, but otherwise reserves all copyrights whatsoever. 

BACKGROUND OF THE INVENTION 
Field of the Invention (Technical Field): 

The present invention relates to evaluation of audience attention and response to advertising in 
video, printed, and on-line formats. 



COPYRIGHTED MATERIAL 
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SUMMARY OF THE INVENTION (DISCLOSURE OF THE INVENTION) 
The present invention is of an automated method of and system for collecting and displaying 
audience recognition information concerning a video presentation, comprising: displaying a video 
presentation to a plurality of subjects; inquiring of each of the subjects by computer means whether each 
5 of a plurality of still images from the video presentation are recognized by each of the subjects; for each 
of the images, tabulating a percentage of the subjects reporting recognition of the image in inquiring; and 
automatically generating by computer means a graph comprising each of the images, wherein a position 
of each of the images on the graph is determined by the corresponding percentage generated in 
tabulating. In the preferred. embodiment, displaying and inquiring are performed on a computer local to 
10 each subject, and tabulating and generating are performed on a central computer networked to each 
local computer. Results of inquiring are preferably communicated to the central computer over the 
Internet. 

The invention is also of an automated method of and system for collecting and displaying 
15 audience recognition information concerning a video or graphic presentation, comprising: displaying a 
video or graphic presentation to a plurality of subjects; obtaining by means local to each subject 
audience recognition information concerning the presentation from each of the subjects; communicating 
results of obtaining via network means to a central computer; tabulating results of obtaining for all 
subjects; and generating by computer means a graph or grid comprising at least one image from the 
20 presentation, wherein characteristics of the at least one image are determined by results of tabulating. In 
the preferred embodiment, displaying and obtaining are performed by World Wide Web browsers and/or 
interactive television devices. Obtaining comprises inquiring of each of the subjects whether each of a 
plurality of still images from a video presentation are recognized by each of the subjects, tabulating 
comprises tabulating a percentage of the subjects reporting recognition of each of the images in 
25 inquiring, and generating comprises automatically generating by computer a graph comprising the 
plurality of images, wherein a position of each of the images on the graph is determined by the 
corresponding percentage generated in tabulating. Alternatively, displaying comprises displaying a 
graphic presentation for a duration of less than one minute, obtaining comprises inquiring of each of the 



plurality of subjects what portions of the presentation are recognized, tabulating comprises determining 
for each of the portions a percentage of the subjects recognizing the portion in inquiring, and generating 
comprises automatically generating by computer means the presentation overlaid with a grid, wherein a 
brightness of portions of the presentation are determined by the results of tabulating. 

The invention is additionally of an automated method of and system for collecting in-market 
audience recognition information concerning a video presentation, comprising: displaying a video 
presentation to a first plurality of subjects; inquiring of each of the first plurality of subjects whether each 
of a plurality of still images from the video presentation are recognized by each of the first plurality of 
subjects; for each of the images, tabulating a percentage of the first plurality of subjects reporting 
recognition of the image in inquiring; generating a graph comprising each of the images, wherein a 
position of each of the images on the graph is determined by the corresponding percentage generated in 
tabulating; after generating, creating an abbreviated presentation containing a subset of the images; after 
introduction of the video presentation via an advertising campaign, displaying the abbreviated 
presentation to a second plurality of subjects in a market targeted by the advertising campaign; and 
inquiring of each of the second plurality of subjects whether the abbreviated presentation is recognized 
by each of the second plurality of subjects. In the preferred embodiment, creating comprises creating a 
storyboard comprising still images or an abbreviated video presentation comprising a subset of the video 
presentation. 

Objects, advantages and novel features, and further scope of applicability of the present invention 
will be set forth in part in the detailed description to follow, taken in conjunction with the accompanying 
drawings, and in part will become apparent to those skilled in the art upon examination of the following, 
or may be learned by practice of the invention. The objects and advantages of the invention may be 
realized and attained by means of the instrumentalities and combinations particularly pointed out in the 
appended claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
The accompanying drawings, which are incorporated into and form a part of the specification, 
illustrate several embodiments of the present invention and, together with the description, serve to 
explain the principles of the invention. The drawings are only for the purpose of illustrating a preferred 
embodiment of the invention and are not to be construed as limiting the invention. In the drawings: 

Fig. 1 is a graph showing the relationship between spending and efficiency in gaining awareness 
for a large number of television commercials of a single organization; because an audience recognition 
measure is,used in the present invention, "wearout" for the same advertisement can be measured over 
time as a function of pending in market, rather than estimated by a theoretical model; 

Fig. 2 is a sample Flow of Attention® graph of percentage of an audience recalling an image from 
a television commercial; 

Fig. 3 illustrates Flow of Attention® information used to perform in-market tracking (prior art); 

Fig. 4 illustrates another type of Flow of Attention® display for print advertising (prior art); 

Fig. 5 is a block diagram of the preferred apparatus of the invention; 

Fig. 6 is a flow diagram of the preferred automated process of collecting and displaying audience 
attention information according to the invention; and 

Fig. 7 is a flow diagram of the preferred Internet-enabled process of the invention for collecting 
audience attention information. 
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DESCR1PTION OF THE PREFERRED EMBODIMENTS 
(BEST MODES FOR CARRYING OUT THE INVENTION) 
The present invention is of an automated process and system for measurement and the display 
of measures of audience attention to motion pictures or video, such as television commercials. The end 
product of this process is a graphical user interface display of research data referred to by the 
trademarks Picture Sorts® and Flow of Attention® (see Fig. 2). Hand-collected and produced 
information displays of this kind (and of the Flow of Attention® display of Fig. 4) have been known in the 
art from previous efforts of Applicant and licensees. 

The automated process of the invention begins with a set of images or key frames taken as a 
sample of the visual information contained in the video stimulus - i.e., the motion picture or video being 
researched. The number of pictures is preferably determined by the visual complexity of the video 
stimulus and is not determined by a mechanical rule or predetermined time sequence. The preferred 
procedure for sampling the video is to take as many pictures as a skilled judge can distinguish in a 
meaningful way. 

The sample of images taken forms an esthetic "vocabulary" for describing how an audience 
cognitively processes the visual information embedded in the symbol system of the video. This 
vocabulary is used to interrogate members of the audience or respondents about the visual experience 
they had from the video. 

Referring to Fig. 5, in the preferred apparatus of the invention 10 images are brought up 
automatically, in randomized order, on a computer or television screen 14 and respondents are asked to 
indicate which pictures or images they recall or do not recall seeing in the video via an input device 16 
such as a keyboard or mouse. The image display and results tallying is preferably performed by local 
computer 12. Alternatively, network-accessible computer 20 can be employed via network connection 
18 (e.g., an Internet connection) to generate images and/or tally results from one or more local 
computers 12. As is readily understood by one skilled in the art, World Wide Web ("web") server and 



browser software can be employed on computers 20 and 12, respectively. This simple 
affirmative/negative, non-verbal method allows researchers to reach across language barriers to explore 
the "esthetic" communication of the video as defined by A. Moles, Information Theory and Esthetic 
Perception (Urbana, University of Illinois Press, 1968). The preferred automated process of the 
invention is shown in Fig. 6. 

The automated process of the invention then uses computer software on local computer 12 
and/or network-accessible computer 20 to accumulate responses across different members of the 
audience to display the results in the form of a Flow of Attention graph. The automated display can be in 
near real time to the period of data collection and/or or be available for subsequent printing, either in 
electronic or paper form, e.g., for use in a report to the creative managers of the video. 

In the automated Flow of Attention® graph of Fig. 2, the pictures are plotted from left to right (on 
an x-axis) in the order in which they actually appear in the video. The pictures are plotted at different 
heights on the y-axis, according to the percentage of the audience who recalls that particular image. For 
the automated process, certain rules for displaying the pictures are used. Preferred rules include, but 
are not restricted to, the following: a) The height of the picture is measured from the top of the picture, b) 
X (a parameter) number of pictures will be displayed per screen or page, and c) when two pictures 
overlap, then the picture with the higher score will be displayed on top if the score is above the average 
for the set of pictures; but if the score is below the average score, then the picture with the lower score 
will be displayed on top. 

The resulting graph forms a user-friendly interface that can be used by creative managers of the 
video to better understand how audiences attend to the flow of images in the video. This information 
may then be used as input to the editing process for optimizing performance of the video. 

The invention is also of a method and system for automatically collecting data on audience 
attention to the esthetic elements comprised in a video communication stimulus, such as a television 



advertisement or training video, or a graphical communication object, such as a print advertisement or 
web page. 

The object being studied should be viewed as a system of symbols and the object can be 
deconstructed or decomposed into discrete esthetic elements which can be used to form a natural 
"vocabulary" for describing and measuring audience members' attention to and response to different 
elements of the symbol system of the communication object. 

In the case of a video stimulus, the vocabulary that is created to describe the system of symbols 
are visual frames as a sample of the total visual information content of the esthetic object. The number 
of frames used is not determined by a mechanical rule, such as after every fixed number of seconds, but 
rather is determined by the esthetic judgment of a skilled interpreter who can identify meaningful 
differences between sampled frames. (See Fig. 2.) 

In the case of a graphic display object (see Fig. 4), the elements are determined by describing 
and measuring audience response to the graphic object after controlled time exposures via a 
computerized interview. For example, the respondent might be automatically exposed to the test for 
three time exposures: 1 / 2 second, 1 second, and 4 seconds. After each exposure, respondents are 
asked to indicate where exactly on the page or screen he or she saw items and that data is recorded on 
a grid or recorded as a probability density function on a screen. Respondents are also asked to record 
what they saw. The information is then coded to determine how long it takes for viewers to register key 
elements such as a headline, a character, a package, a brand name, or the like. Depending on the 
purposes of the study, different time periods and a different number of exposures may be used. This 
method of data collection measures the order in which respondents take in or process the esthetic 
information contained in a display object. 

Unlike other research methods for testing communication objects, which rely on verbal methods 
(i.e., "semantic" information, as described by Moles in Information Theory and Esthetic Perception, 



supra), this method relies on the "esthetic" vocabulary of the communication object itself to describe and 
measure how audiences cognitively and emotionally process the information contained in the 
communication object. Of necessity, therefore, this approach requires that visual esthetic objects be 
placed before the eyes of audience members or research respondents. 

The Internet and interactive television make possible the collection of data on audience attention 
to esthetic objects utilizing recognition-based as opposed to recall-based research measures. The 
invention, again referring to Fig. 5 and also to Fig. 7, utilizes apparatus and software written to operate 
on Internet World Wide Web browsers or interactive television devices to expose "samples" of 
communication objects for use as a natural esthetic vocabulary in research data collection methods. 
These samples consist either of deconstructed video elements (i.e., frame grabs) or samples "in time" 
(i.e., controlled time exposures). These can then easily be manipulated by computer technology to 
interrogate, from a remote distance, an audience's response to a communication object in the natural 
environment of the home or place of business rather than in the artificial environment of a research 
center. 

The invention is further of an automated system and method for using the Internet or interactive 
television for tracking audience awareness and response to television advertising. The method uses a 
recognition-based approach to research measurement as opposed to the recall-based approach used by 
many other research companies to track television advertising awareness. The importance of this 
approach is that it makes it possible for an advertiser to track the awareness of an individual 
advertisement embedded in a larger pool of television advertisements being aired simultaneously. This 
approach also makes it possible for an advertiser to track the "wear out" of individual commercials in real 
time (see Fig. 1), as opposed to current methods which use estimates for theoretical models. 

Recognition of the television commercial, as a measure of audience awareness, is obtained by 
showing an audience member or respondent a storyboard (photoboard) of the advertisement or 



abbreviated video on a computer or television screen (see Fig. 5). This can be done via the Internet or 
interactive television. 

The storyboard/abbreviated video approach makes use of a reduced-set of esthetic information 
from the original video in that only a small number of frames from the advertisement are included in the 
storyboard. However, research has shown that a viewing audience will give different levels of attention 
to different images in a television commercial (e.g., C. Young, et al., "Video Rhythms and Recall", 
Journal of Advertising Research 29, 3, pp. 22-25 (1989)). Therefore, without prior information as to 
which images in an advertisement are most memorable, the choice of images used in constructing a 
storyboard can severely affect the accuracy of the recognition-based measure obtained from use of a 
storyboard or abbreviated video stimulus. 

A filtering process must first be used in the choice of images selected for the storyboard. The 
filtering process used in this invention is to use the Picture Sorts® method, as employed in the Flow of 
Attention® graph of Fig. 2, to identify key images for use in the in-market tracking research. In other 
words, images scoring above some predetermined hurdle rate in the "forced-viewing" situation of a 
commercial pre-test, which measure show consumers cognitively process or attend to television image 
a commercial in a non-cluttered environment, are those which are included in the storyboard or 
abbreviated video stimulus which is used to track attention to the commercial in the cluttered 
environment of the real world marketplace. (See Fig. 3.) 

The effect of this filtering process is to reduce the measurement error around the storyboard- 
based recognition measure. As a result, the system for tracking audience awareness and response to 
television advertising is more accurate than it otherwise would be. 
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Although the invention has been described in detail with particular reference to these preferred 
embodiments, other embodiments can achieve the same results. Variations and modifications of the 
present invention will be obvious to those skilled in the art and it is intended to cover in the appended 
claims all such modifications and equivalents. The entire disclosures of all references, applications, 
patents, and publications cited above are hereby incorporated by reference. 



